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ABSTRACT 

In this paper, the bibliographical references in five 
core entomology journals are examined for citation accuracy in order 
to determine if the error rates are similar. Every reference printed 
in each journal's first issue of 1992 was examined, and these were 
compared to the original (cited) publications, if possible, in order 
to determine the accuracy of the reference. Percentage of total 
errors was calculated for each journal, and the errors were sorted by 
type, such as an error in an author's name or in pagination. The 
study focused on the following research questions: (1) "What 
percentage of citation errors typically occur in the most cited 
entomology periodical literature? Overall, are the references in 
these entomology journals more or less accurate than those in medical 
and library science journals?"; (2) "What types of errors typically 
occur, and how do they compare with results of other studies?"; and 
(3) "What types of publications do authors make the most mistakes in 
citing?" It was found that, on average, 30 percent of citations in 
the journals contained one or more errors. Most of the errors 
appeared in journal article titles, with article author names and 
pagination errors being the second and third most frequent. Some 
types of errors occurred very infrequently; these included: journal 
title, volume, page and year omissions, and book publication year 
omissions. Out of the 49 articles in the 5 journal issues examined, 
only 3 articles had completely error-free reference lists. (Contains 
24 references.) (AEF) 



it it it it it it it it it it it i .- it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it Vi it it it it it it it it it it it it it it it it it it it it it it it it it it 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

it it it it it it it it it it it it it it it it it it it it ,': it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it 



U S DEPARTMENT OF EDUCATION 
EDUC -riONAL RESOURCES INFORMATION 

CENTER (ERIC) 
q This document has been icpioduced as 

received trom the person or organization 

originating it 
□ Minor changes have been made to 

improve reproduction quality 

• Points of view or opinions sta.ed in this 
document do not necessarily represent 
official OERI position or policy 



ACCURACY OF REFERENCES IN FIVE ENTOMOLOGY JOURNALS 



A Master's Research Paper submitted to the 
Kent State University School of Library and Information Science 
in partial fulfillment of the requirements 
for the degree Master of Library Science 



by 

Cynthia Kristof permission to reproduce this 

MATERIAl HAS BEEN GRANTED BY 

May , 1 995 _ fJ.? n K r 1 H } °£ 

BEST COPY AVAILABLE 

TO THf EDUCAHONAI. RESOURCES 
INI OHMATION i.[-NTFR it RICi 1 



Master's Research Paper by 
Cynthia Kristof 
B.A., The Ohio State University, 1989 
M.L.S., Kent State University, 1995 



Approved by 



Adviser Date_^f^_ff i!?? 5 ' 



Er|c 3 



Studies of the accuracy of bibliographical references have been done in 
medical and library science journals. Every study performed so far has detected 
errors, such as incorrect volume or page numbers, inaccurate article or journal 
titles, and misspelled authors' names, in the bibliographical references examined. 
Bibliographical reference accuracy is essential to the success of scholarly 
communications, bibliometric studies, citation indexes, databases, and the 
interlibrary loan and document delivery processes. However, such a study of the 
bibliographical references in entomology periodicals has not yet been published. 
What types of reference errors occur in citations in entomology journals? How do 
these errors compare by rate and by type? What types of publications do authors 
make the most mistakes in citing? In this study, the bibliographical references in 
five core entomology journals are examined. Every reference printed in each 
journal's first issue of 1992 was examined, and these were compared to the original 
(cited) publications, if possible, in order to determine the accuracy of the reference. 
Percentage of total errors was calculated for each journal, and the errors were sorted 
by type, such as an error in an author's name or in pagination. It was found that, on 
average, 30% of citations in the entomology journals studied contained one or more 
errors. 
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Introduction: 

The problem of inaccurate bibliographical references is one that has existed 
for as long as authors have cited other authors. The most famous case of an 
inaccurate bibliographical reference citation is the case of "Dr. O. Uplavici." "O 
uplavici" actually refers to amoebic dysentery; however, an article written by Czech 
medical professor Jaroslav Hlava in 1887 on amoebic dysentery, got mistakenly 
attributed to a "Dr. O. Uplavici." This error was repeated in various forms for years 
until Clifford Dobell wrote the story of this famous foul-up in 1938 (Dobell, 1938). 
This humorous example of bibliographical citation error illustrates two things: one, 
that citation error occurs, even when an article's author is a respected researcher, 
. and it is more common than one would think; and two, that citation error can self- 
perpetuate, become embarrassing, and can hinder scholarly communications. 

Indeed, several authors have pointed out that bibliographical references are 
used for many things. References are often "called upon to locate papers" not yet 
indexed (Place, 1916, p. 697); they "provide a method of evaluating the article; [and] 
assist in assessing an author's credibility" (Foreman and Kirchhoff, 1987, p. 177). 
Furthermore, accurate references are necessary to the success of bibliometric studies, 
citation indexes and databases, and the interlibrary loan and document delivery 
processes (Pandit, 1993). Many respected authors do not actually check the primary 
document before citing it, but rather take "a reference from another's bibliography 
as though it were thereby Gospel truth itself" (Place, 1916, p. 699). This explains why 
"Dr. O. Uplavici" was listed in the 1910 Index-Catalogue of Medical and Veterinary 
Zoolog y as having received a degree! 

At present, studies of citation accuracy have been performed mainly in the 
fields of medicine and library science. Studies in the fields of education, 
psychology, social sciences, business, and other areas are lacking. One such area is 
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the biological sciences. This study examined five core entomology journals for 
citation accuracy in order to determine if the error rates are similar. 

Literature Review: 

In Doms' 1989 study of five dental journals, he defined two categories in 
which all references may be divided: "correct or incorrect. A correct reference was a 
reference that was identical to the source. An incorrect reference was a reference 
that deviated from the source" (p. 442). Errors, in Doms' study, "involved names or 
initials of authors, title of article, name of journal, volume number, year of 
publication, page numbers, punctuation, [and] spelling..." (p. 442). Yankauer (1990) 
defines citation errors as, "errors of commission or omission in the printing of the 
reference" (p. 38). Pandit states that errors "focus on the citations themselves and 
exclude the extent to which authors correctly quoted a text or acknowledged an 
intellectual debt" (p. 185). 

When defining what would be counted as an "error", all studies have looked 
for common elements, such as author, journal title, volume, and year, that journals 
usually require for contributing authors' bibliographical references. For example, 
Key and Roland (1977) defined seven categories of errors: author, title, journal, 
volume, pagination, year, and publisher. If an author happened to give 
"superfluous information" (p. 136), such as the month of publication, than this was 
deleted and not counted as an error. Deviations from the stylistic requirements of 
the journals, such as the order of the volume, pagination, and year, or incorrect 
Index Medicus abbreviations, were not counted as errors, either. Poyer (1979) chose 
not to include an error category for article title, since "many journals do not require 
titles, and if there was a mistake (for example, spelling or omission), it would be 
impossible to decide how this would affect the potential use of the cited document" 
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(p. 397). Pandit included five error categories: name of the journal/book, title of the 
work, author /editor, volume and issue number, pagination, and year, and 
publisher. Pandit's study differs from most previous studies in that she included 
wrong and missing issue numbers as errors "except in the case of journals which 
follow the policy of including seasons or months instead of issue number" (p. 190). 
She "compensated for the fact that for journals using the American Psychological 
Association (AP A) style or similar styles, the issue number is not used unless each 
issue begins with page 1" (p. 190). No studies thus far have addressed any other 
possible categories of errors, such as monographic series titles. 

Benning and Speer maintain that library science literature contains fewer 
"major" errors, which "prevent immediate identification of the source of the 
reference," (de Lacey, 1985, p. 884) than does medical literature. Many studies in 
both medicine and library science discuss "major" and "minor" errors and the 
differences between the two. Doms (1989) defines "minor" errors as "omissions that 
did not prevent the location of the article, and included: paraphrased or incomplete 
article titles, incorrect author initials, or an error in the last page number of an 
article" (p. 442). Doms defines "major" errors as "references that prevented 
immediate location of the article cited, and included: incorrect journal titles, article 
titles, author and citations such as wrong volume, issue, year, and first page 
numbers" (p. 442). Many other studies of both medical and library science literature 
have categorized citation errors as either "major" or "minor" (Benning and Spear, 
1993; de Lacey, Record, and Wade, 1985; Eichorn and Yankauer, 1987; Evans, Nadjari, 
and Burchell, 1990; Foreman and Kirchhoff, 1987; Pope, 1992; Putterman and Lossos, 
1991; George and Robbins, 1994; Goldberg, Newton, Cameron, Jacobson, Chan, 
Bukata, and Rakab, 1993; Hinchcliff, Bruce, Powers, and Kipp, 1993; and Nuckles, 
Pope, and Adams, 1993). However, Pandit's study does not classify errors in terms of 
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"major" or "minor," nor do some other studies (Key and Roland, 1977; Stull, 
Christina, and Quinn, 1991; and Poyer, 1979). Sweetland (1989, p. 296) points out 
that: 

...a 'slight misspelling' in the author's last name is a major error for the user 
of the ISI citation indexes, where this name is the primary access point, 
while it is trivial if the user is looking up the reference in the actual 
journal. 

Most studies which chose to classify errors as "major" or "minor" categorize name 
misspelling errors as "minor." Hinchcliff, Bruce, Powers, and Kipp (1993) classified 
name misspelling errors as "major" however, noting that they "would likely 
impede the rapid, computerized retrieval of the cited material or had the potential 
to obscure the identity of the author(s)" (p. 398). Goodrich and Roland state in their 
1977 article that they did not "attempt to make value judgments" by assigning such 
labels as "serious" and "trivial"; they say that "most of the errors they found were 
sufficient to require the expenditure of extra time to resolve the discrepancy" (p. 19). 

Many studies have selected journals for study because of the journals' subject 
area and scope; however, a few studies have made an attempt to select "core" 
journals in a subject area. Poyer, in his 1979 study of significant science journals, 
chose to study 34 of the "206 most cited and 206 high-impact journals," (p. 396) as 
judged by Eugene Garfield in his 1976 article, "Significant Journals of Science," 
( Nature 264: 609-15). Pope (1992) attempted "to cover several of the subfields of 
study within librarianship and of the divisions of the American Library 
Association" (p. 240) in choosing her ten library science journals. Pandit (1993) 
selected the journals for her study by consulting "the ranked list of journals in ISI's 
Tournal Citation Reports and selected studies that have identified core journals" (p. 
189). 

Several studies thus far have used similar sampling techniques when 
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choosing which bibliographical references to examine in the journals studied. De- 
Lacey, Record, and Wade (1985) examined fifty references, chosen by random 
numbers, out of the first issues in 1984 of the journals they chose to study. They 
examined quotations from "original articles, reviews, leading articles, and 
correspondence" (p. 884). Eichorn and Yankauer (1987) examined fifty randomly 
chosen references to journal articles only, from the May 1986 issues of the journals 
they chose. Doms (1989) studied 500 randomly selected references from the March 
1987 issues of chosen dental journals. Evans, Nadjari, and Burchell, in their 1990 
study, similarly chose fifty random references from the August 1987 issues of 
surgical journals. Pope, in her 1992 study, examined ten references each from the 
first 1991 issues of ten library science journals. Benning and Speer (1993) examined 
"all 555 references found in the articles in the final 1989 issues" (p. 56) of the library 
sciences journals they studied. Finally, Nuckles, Pope, and Adams, in their 1993 
stu ly of ten dental journals, randomly selected thirty references from the January 
1991 issues (p. 28). 

All studies thus far have made an attempt to verify references directly, by 
comparing the published citation against the original source. For Pandit's (1993) 
study, this proved wholly successful. When comparison with the original source 
was impossible, usually due to the researchers' libraries not containing the cited 
documents, researchers generally took one of two approaches. Many studies made 
an attempt to obtain the either a hard copy of the item or a photocopy through 
standard interlibrary loan (McLellan, Case, and Barnett, 1992; Evans, Nadjari, and 
Burchell, 1990; de Lacey, Record, and Wade, 1985; and Hinchcliff, Bruce, Powers, and 
Kipp, 1993). De Lacey, Record and Wade (1985) "classified as unverifiable" (p. 884) 
any item that could not be obtained through interlibrary loan. Other studies opted 
to utilize indexes, databases, or other similar sources, such as union lists and Books 



in Print to verify references when no hard copy or photocopy was available 
(Foreman and Kirchhoff, 1987; Poyer, 1979; Boyce and Banning, 1979; Key and 
Roland, 1977; and Doms, 1989; Benning and Speer, 1993; McLellan, Case, and 
Barnett, 1992). Pope (1992) points out that "an index has the potential to be as 
unreliable as the citations themselves" (p. 241), although four of the references she 
examined were verified using an index. Three studies did not include references 
for which the original source was not owned by the library (Goodrich and Roland, 
1977; Putterman and Lossos, 1991; and Eichom and Yankauer, 1987). Only two 
studies classified unverifiable references as erroneous (Doms, 1989; and Pope, 1992). 

The results of the bibliographical citation studies in medicine have been 
surprising, and remarkably similar. One of the first studies, by Goodrich and 
Roland in 1977, found that among ten respected medical journals, 29% of 
bibliographical references studied were found to be erroneous. A study of medical 
and other science journals by Poyer in 1979 showed that out of 2,448 references 
verified, 466 errors were found, which involved 367 references (15% of all 
contained at least one error). De Lacey, Record, and Wade, in their 1985 study of six 
medical journals, found that citation error rates in the journals r.tudied ranged from 
8% to 46%, averaging 24% overall. A 1987 study of nursing journals by Foreman 
and Kirchhoff found that in clinical nursing journals, 38.4% of cited references were 
erroneous, and that in nonclinical nursing journals, 21.3% of cited references were 
inaccurate. In Doms' 1988 survey, it was found that 42% of the references examined 
were incorrect. A study of four anesthesia journals, by McLellan, Case, and Barnett 
in 1992, found that 50.3% of references contained at least one error. A study of three 
public health journals by Eichorn and Yankauer found that, overall, 31% of 
contained either a "major" or "minor" error. A study of surgical journals by Evans, 
Nadjari, and Burchell showed an overall error rate of 48%. A study of veterinary 



medicine journals by Hinchcliff, Bruce, Powers, and Kipp in 1993, revealed an error 
rate of 30%. Stull, Christina, and Quinn, in their 1991 study of Research Quarterly 
for Exercise and Sport found that 47% of that journal's examined citations were 
erroneous in some way. Goldberg, Newton, Cameron, Jacobson, Chan, Bukata, and 
Rakab's 1993 study of emergency medicine literature revealed an average error rate 
of 28%. A survey of ten dental journals by Nuckles, Pope, and Adams (1993), found 
that 21% of citations were incorrect. George and Robbins' 1994 study of the 
dermatologic literature showed that 41% of the verified references contained at least 
one error. Apparently the situation is similar overseas; a study of the two major 
Israeli medical journals in 1991 by Putterman and Lossos revealed that only two- 
thirds of references examined were error-free (33% contained errors). 

The overall situation is only slightly better for the field of library and 
information science, which is surprising since librarians are often acquainted with 
the consequences of inaccurate references. A 1979 study by Boyce and Banning 
which examined references in the Tournal of the American Society for Information 
Science and Personnel and Guidance Tournal showed that references in the former 
contained 13.6% total errors, and references in the latter contained 10.7% total 
errors. A 1992 study by Pope of ten library science journals revealed that 30% of 
bibliographical references contained either "major" or "minor" errors. Pandit's 1993 
study of five library science journals showed that out of 1,094 references, 223 errors 
were detected in 193 of the references (18% of the references contained at least one 
error). Finally, Benning and Speer's 1993 comparison of library literature ( Bulletin 
of the Medical Library Association, Library Resources and Technical Services , and 
Library Trends ) with medical literature (studies by de Lacey, Evans, and Eichorn), 
found tha* 29% of examined references in library science journals contained either a 
"major" or "minor" error. 
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Some journals, such as Archives of Physical Medicine and Rehabilitation and 
Library Trends, have review staff who check the accuracy of references in articles 
submitted for publication. Key and Roland (1977) examined 129 articles accepted for 
publication by the Archives of Physical Medicine and Rehabilitation . They found 
that 54% of the 1,867 references examined were incorrect, and 6% could not be 
verified. Key and Roland did not study the same articles after they were printed; 
however, they discuss the results of the study by Goodrich and Roland (1977) and 
surmise that "several of the journals they studied must have followed the practice 
of checking references; periodicals such as the Tournal of Bone and Toint Surgery, 
with an error rate of 13%, compare highly favorably with journals such as TAMA, 
with a rate of 50%" (p. 137). Pandit's 1993 study of five library science journals 
includes an examination of the journal Library Trends whose editorial staff 
"verifies references against the original sources" (p. 193). Pandit found that, out of 
the 335 references examined, 177 (52.8%) contained errors at the manuscript stage, 
and just 14 (4.2%) errors were present in print. The presence of review staff who 
check references at the manuscript stage seems to significantly reduce the amount of 
errors present in print. 

The types of citation errors found in the studies have also proven remarkably 
consistent. Studies by Poyer (1979), Benning and Speer (1993), Evans, Nadjari, and 
Burchell (1990), Eichorn and Yankauer (1987), Key and Roland (1977), Putterman 
and Lossos (1991), Stull, Christina, and Quinn (1991), Pope (1992), Hinchcliff, Bruce, 
Powers, and Kipp (1993), and George and Robbins (1994), showed that errors are 
most commonly found in authors' names. Errors in authors' names included 
misspellings and partial or total omissions in both surnames and initials. Errors in 
article titles, which included "paraphrasing or shortening of article titles" (Pope, 
1992, p. 241), ranked second in most of these studies, an exception being Poyer 's 
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(1979) study, which omitted article title verification from the study. Four studies 
(Doms, 1989; Goodrich and Roland, 1977; McLellan, Case, and Barnett, 1992; and 
Nuckles, Pope, and Adams, 1993) showed that article title errors were most 
common, followed by errors in authors' names. Two studies (de Lacey, Record, and 
Wade, 1985; and Evans, Nadjari, and Burchell, 1990) which did not rank types of 
errors numerically, revealed errors most often in article titles and authors' names. 
Studies by Pandit (1993) and Boyce and Banning (1979) revealed that most errors 
were numerical in nature, occurring in volume numbers, issue numbers, page 
numbers, etc. Thus, no real generalizations can be made regarding frequency, of 
different types of errors between medical and library science literature; all types of 
journals seem to be prone to all types of errors. 



Research Objectives and Hypothesis: 

Since no bibliographical reference accuracy study had been done on the 
periodical literature in the field of entomology, this study examined five core 
entomology journals and attempted to answer the following research questions: 

1. What percentage of citation errors typically occur in the most cited 
entomology periodical literature? Overall, are the references in these 
entomology journals more or less accurate than those in medical and library 
science journals? 

2. What types of errors typically occur, and how do they compare 
with the results of other studies? 

3. What types of publications do authors make the most mistakes in citing? 
Are book chapters, for example, more problematic for authors to cite than are 
regular journal articles? 

Thirty-one percent L he combined average error rate of all the studies, in both 
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medical and library sciences, reported in the Literature Review section of this paper. 
How do entomology journals compare? The hypothesis of this study: that 
entomology journals have a 31% error rate as well. 

A "correct", or "accurate", citation is defined as one in which all included 
elements are "identical to the source" (Doms, 1989, p. 442). "Errors," for the 
purposes of this study, are defined as "deviations from the source" (Doms, 1989, p. 
442). Errors include both incorrect (including incomplete) and omitted (where it is 
required) citation information. An "article" is defined as a regular research article 
appearing in a journal issue, including "short communications" or "mini-reviews" 
"Types of publications," include cited journal articles and books, including 
government documents, dissertations, and conference proceedings. Patents are 
excluded due to a lack of a standard form of citation. It was assumed that some 
references would be impossible to verify, such as some conference proceedings, 
some dissertations, and all items "in press" at the time the references were 
published. It was assumed, however, that most references would be verifiable. Due 
to varying uses by previous studies quoted in the literature review, it should be 
noted that "citation" and "reference" are used interchangeably and are synonymous 

Methodology: 

In order to answer the research questions and test the hypothesis, an 
exploratory, descriptive study was performed on five entomology journals. The 
type of data collected includes information on the bibliographical references found 
at the end of articles published in the five entomology journals. These references 
were examined for accuracy. 

The journals chosen for this study were selected based upon their "impact" 
ranking in ISI's Science Citation Index Journal Citation Reports from 1992. 
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According to ISI, a ' high impact" journal has the highest "average number of 
current citations to articles [the] journal published in the previous two years." The 
six entomology journals with the highest "impact factor" for 1992 include: 

1. Annual Review of Entomology 

2. Insect Biochemistry and Molecular Biology 

3. Journal of Insect Physiology 

4. Pesticide Biochemistry and Physiology 

5. Archives of Insect Biochemistry 

6. Ecological Entomology 

The highest impact journal, Annual Review of Entomology, was excluded from 
this study due to its nature as an annual review periodical. Previous studies, upon 
which this study is based, did not examine annual periodicals, and the number of 
references in the annual publication would have been disproportionate to the other 
journal issues whicr were examined. Thus, the remaining five on the above list 
were selected for this study. 

The sampling technique used is based upon previous studies of this nature 
(Poyer, 1979; Doms, 1989; Benning & Speer, 1993; Pope, 1992; Evans, Nadjari, and 
Burchell, 1990; Eichorn & Yankauer, 1987; Nuckles, Pope, and Adams, 1993). The 
first issue published in 1992 for each journal was studied, as these journal issues are 
representative of typical issues for these publications. Every reference at the end of 
each article in each issue was examined, excluding references designated "in press," 
(n=15) and patents (n=3). Book reviews and letters to the editor did not appear in 
the issues examined, and therefore were not studied. As in previous studies, each 
journal has a different publication frequency, each issue has a different amount of 
articles, and each article contains a different amount of references. Insect 
Biochemistry and Molecular Biolog y is published 8 times a year; Journal of Insect 
Physiolog y is published monthly; Pesticide Biochemistry and Physiolog y is 
published monthly except for April, August, and December; Archives of Insect 
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Biochemistry and Physiolog y is published monthly; and Ecological Entomolog y is 
published quarterly. Excluding references to patents and "in press" items, in the 
first 1992 issues, Insect Biochemistry and Molecular Biolog y contains 11 articles and 
311 refe, ences, Journal of Insect Physiolog y contains 8 articles and 249 references, 
Pesticide Biochemistry and Physiolog y contains 10 articles and 245 references, 
Archives of Insect Biochemistry and Physiology contains 5 articles and 155 
references, and Ecological Entomology contains 15 articles and 377 references, for a 
total of 1,337 references. The sampling technique might have introduced some 
error; for example, the January 1992 issues may have been, by chance, particularly 
"good" issues for these journals. However, the fact that an attempt has been made 
to examine each reference in each chosen issue, and not just a random sample of 
the citations in those issues, helps to improve reliability. 

Types, or categories, of errors for this study were primarily based on Pandit's 
definitions (1993, p. 190). Author/editor errors are incorrect authors' /editors' 
names, including misspellings, omitted authors' names, and incorrect, reversed, or 
omitted initials. Paraphrased or incorrect titles of articles were counted as errors, 
including omissions or misspellings of words, and mistaken additions or omissions 
of plural forms of words. Incorrect book and journal titles were counted as errors. 
Incorrect, misspelled, or omitted publishers or places of publication for books were 
counted as errors. Incorrect or omitted volume numbers, page numbers, and years 
were also counted as errors. None of the journals require, or included, journal 
issue numbers or months in citations. Two of the journals, Archives of Insect 
Biochemistry and Physiolog y and Pesticide Biochemistry and Physiolog y require 
that authors note only the first page number of journal articles; end page numbers 
are not required. Also, none of the journals required series titles as part of book 
references; therefore, these were not recorded even if they were included. Citations 
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to books in a series were classified as "books" or "journals" depending upon how 
the author cited them. For example, a citation to an article from the series Methods 
in Enzymolog y was classified as a book citation if the author included the publisher 
and place information and constructed it like a book citation; it was classified as a 
journal citation if the author did not include the publisher and place information, 
and configured the citation like a journal citation. 

Deviations from the journals' required styles were not generally counted as 
errors. For example, if a journal requires that contributors use Index Medicus 
journal title abbreviations, and an author made an error in abbreviation, this was 
not counted as an error unless the journal title was incorrect, or unless the 
abbreviation error made identification of the journal title difficult. An example of 
this is the frequent mistaken abbreviation for the journal Biochimica et Biophysica 
Acta as Biochem Biophys Acta . Correct Index Medicus abbreviation for this title is 
Biochim Biophys Acta (boldface emphasis added by the researcher). If any of the 
citation's elements, such as volume, year and pages, were out of the required order, 
this was not counted as an error (Key and Roland, 1977). 

Out of each issue to be studied, two photocopies of the references at the end of 
each article were made. One photocopy was used as a record of the original set of 
references. Each reference was given a letter code to designate from which journal 
and which article the reference originates, and the references were numbered 
consecutively for each article. The second set of photocopies was used to cut out a 
copy of each reference, which was attached to an evaluation sheet (see appendix), 
and given the same code and consecutive number. These evaluation sheets were 
used to evaluate the accuracy of the reference, to keep track of references, and to 
compile the raw data into usable form. This instrument was field tested by 
following the procedure on a sample set of references at the end of one of the 
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articles in one of the journal issues examined for study. Missing elements were 
noted and included in a revised, final version of the evaluation sheet. 

The references were compared with the original piece, or a photocopy or 
facsimile of the original piece, if possible. Errors were recorded specifically either as 
"incorrect or incomplete" or as an "omission." If the piece was checked out, a 
"hold" was placed on the item, which generally made the item available to the 
researcher within ten days. When the Ohio State University Libraries did not own 
the item, either the item itself or a photocopy of the item was obtained through 
interlibrary loan. If the item could not be obtained through interlibrary loan, then 
an attempt was made to verify the reference in indexes or databases. Due the the 
possibility that indexes and databases themselves may contain errors, an attempt 
was made to verify the references in two or more indexes or databases. If index or 
database verification was not possible, the reference was labeled "unverified." 
Some references were impossible to verify; these were some conference 
proceedings, some dissertations and rare, foreign-language publications unavailable 
through interlibrary loan, and some "unidentifiable" publications. These 
"unidentifiable" ones could not be found in any database. This made interlibrary 
borrowing impossible, as well as database or index verification. Any unverifiable 
references were counted in the total number of references examined, but were not 
considered errors, nor were they used to calculate error rates or percentages of 
errors. (See Table 1.) 

The type of publication cited was also noted on the evaluation sheet, whether 
it was a book or book chapter, or a journal article. This was used to determine 
which of the two types of publications, books or journals, are most difficult for 
authors to cite correctly. 
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Table 1 : Unverifiable Citations 

Journal Title Abbreviation 

Reason 





AP 


m 


B 


JP 


PB 




Conference proceedings 


0 


0 


1 


1 


3 


5 


ILL unavailable - Dissertation 


0 


2 


0 


0 


0 


2 


ILL unavailable - Regular 


0 


1 


0 


0 


0 


1 


Unidentifiable 


1 


3 


1 


0 


1 


6 


Totals 


1 


6 


2 


1 


4 


1 4 



Key: AP=Archives of Insect Biochemistry and Physiology, EE=Ecological Entomology, 
IB=lnsect Biochemistry and Molecular Biology, JP=Journal of Insect Physiology, 
PB=Pesticide Biochemistry and Physiology 
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Results and Discussion: 

In the first issues in 1992 of the top five entomology journals, 518 errors, 
involving 398 citations, occurred in 1,323 verifiable stations. The overall error rate 
is 30%, slightly below the average error rate of 31%. This is not a statistically 
significant difference at the 0.05 level (t=0.21). The journal Ecological Entomology 
had the lowest percentage of errors and Insect Biochemistry and Molecular Biolog y 
the highest error rate of 35%. (See Table 2.) 

Most of the citations (1,309 or 99%) were verified by comparing the citation to 
the original piece, or a photocopy or facsimile of the original piece obtained through 
interlibrary loan. The remaining 14 citations were verified in online databases or 
library catalogs. Ten of these were verified in two or more databases or catalogs; 
however, four of them were verified as being correct in only one database. Three of 
these four were dissertations, and the remaining one was a booklet published by a 
small, local park. 

Most of the errors found in the entomology journals studied appeared in 
journal article titles. (See Table 3.) Thirty percent. (n=157) of the 518 total errors 
found were journal article title errors. Most of these errors seem very minor; often, 
the plural form of a word in the title was used, or not used, by mistake. Words were 
omitted, added, misspelled. Sometimes scientific names were omitted. Errors in 
punctuation within phrases sometimes occurred. This seems very minor, and most 
of the time it causes few retrieval problems. However, consider the difference in 
meaning between these two phrases: "...ascorbate-free radical..." (found in an 
incorrect citation) versus "...ascorbate free-radical..." (found in original publication). 
A simple typing error in a seemingly small word can change the whole meaning of 
a phrase. Consider the difference in meaning between the following two article 
titles: "Dclta-endotoxins from cation-selective channels..." (found in incorrect 
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Table 2: Entomology Journals Studied, Citations, and Errors 






Journal Title 


Total Citations 


Total Verified 


Total # Errors Total Citations with error(s) 


% Errors 


Arch Insect Biochem Physiol 


1 55 


1 54 


70 


43 


28% 


Ecological Entomology 


377 


371 


115 


98 


26% 


Insect Biochem Molecular Biol 


31 1 


309 


141 


108 


35% 


Journal of Insect Physiology 


249 


248 


109 


o U 


32% 


Pesticide Biochem Physiol 


245 


241 


83 


69 


29% 


Totals 


1,337 


1 ,323 


518 


398 


30% 
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citation) versus "Delta-endotoxins form cation-selective channels..." (found in 
original publication). Less often, article titles were paraphrased; this could cause 
delays when attempting to locate an article, by article title, online. An example of 
this is the incorrectly cited article title, "Interaction of fluridone with phytoene 
desaturase of Aphanocapsa." The correct version, found in the original publication, 
is, "Interference of fluridone with the desaturation of photoene by membranes of 
the cyanobacterium Aphanocapsa." 

Errors in names of journal article authors was the second most frequently 
occurring error. Twenty-eight percent (n=146) of all 518 errors were journal article 
author name errors. These errors included omitted, reversed, or incorrect initials, 
and misspelled names. An example of the latter: "D. L. Coppage" was cited as "D. L. 
Coopage." Sometimes completely incorrect authors had been added to citations, 
however, this was relatively rare. As with some article title errors, these errors can 
impede location of an article, especially if a reader of a bibliography writes down or 
memorizes only the name of an author, with the intention of locating that article 
later. Omissions of article authors' names was very infrequent (n=9, 2%); however, 
the consequences are similar. 

The third most frequent error found in this study, though much less frequent 
than journal article title or author errors, was page number errors. These errors 
occurred 35 times, and comprised 7% of all errors. Often, these errors seemed to be 
typographical errors, such as in the example of "389-389"; the correct page numbers 
were 381-389. The type of page number errors that were the most troublesome 
when attempting to locate an article were the ones given, for example, as "1185" 
instead of "1885." Incorrect /incomplete or omitted book chapter page numbers 
(n=18, 3% and n=7, 1%, respectively) can cause similar problems; however, it is 
often easier to look up pages to a book chapter in a table of contents than it is to look 
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up article pages in a journal's yearly index. 

The fourth most common error occurred in book citations: incorrect or 
incomplete book authors' or editors' names (n=27, 5%). Like errors in article 
authors' names, these errors can cause problems when attempting to locate a 
publication in an online database or library catalog. 

Incorrect or incomplete journal titles (n=18, 3%) or book titles (n-=13, 3%), can 
cause serious problems when attempting to locate a journal or a book in a library 
catalog. A simple title-word reversal caused this problem; Tournal of Medical and 
Veterinary Mycology was cited as Tournal of Veterinary and M edical Mycology. A 
few times, the journal Nature was cited instead of the correct Science , and vice- 
versa. These two titles happened to be juxtaposed in the library used by the 
researcher, thus preventing these errors from becoming time-consuming. 
However, this error would have otherwise required the use of an index for 
verification. Another time-consuming title error was an incorrect citation of the 
1986 book Proteinase Inhibitors as Protease Inhibitors . 

Omission of the place of publication of books occurred ten times (2%). While 
this did not cause problems in locating the books if owned by the library, it could 
cause problems when attempting to order books. Another problematic error, which 
occurred 9 times (2%), was the omission of a volume number when a volume of a 
monographic set was cited. In every case, this required the time-consuming process 
of looking in the index to the monographic set. 

Incorrect, incomplete, or omitted volumes and years in journal article 
citations did not cause time-consuming problems in this study, simply because 
these types of errors never occurred simultaneously in the same citation. However, 
both of these types of errors can cause problems for the reader of a bibliography who 
tends to memorize or write down one and not the other. Volume and year errors 
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can also cause problems when a journal is published in more than one volume per 
year. 

Some types of errors occurred very infrequently; these include: journal title, 
volume, page, and year omissions, and book publication year omissions. Some 
types of errors did not occur at all. These were article title omissions, chapter title 
omissions, book publisher errors of both types, and incorrect book volumes. The 
reason for this seems to be the relative "obviousness" of this data. The one-time- 
each occurrence of the journal title, volume, page, and year omission seems to be a 
publisher's error, since they occurred in the same citation. The lines of type 
containing this information simply seem to have been mistakenly left out of where 
they belonged and placed at the end of another line of type, where they were later 
found. 

The types of errors found in this study are very similar to the types of errors 
found in previous studies. Although seven previous studies found most mistakes 
in authors' names, four studies found that article titles contained the most errors, as 
this one did. (See Table 4.) In each case, mistakes in authors' names were the next 
most frequently occurring error, followed by mistakes in numerical citation data. 

Although fewer books were cited than journal articles and most of the errors 
occurred in citations to journal articles, a greater percentage of citations to books 
contained errors than citations to journal articles. For example, in Insect 
Biochemistry and Molecular Biology a full 61% of citations to books contained one 
or more errors versus 32% of journal article citations. Archives of Insect 
Biochemistry and Physiology was the only exception in this comparison; only 21% 
of citations to books contained at least one error, compared to 29% of journal article 
citations. (See Table 5.) It is possible that the reason why citations to books more 
often contain one or more errors is because book citations often contain more 
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Table 4: 
Author 

Bennlng & Speer 
Boyce & Banning 
de Lacey et al. 
Doms 

Eichorn & Yankauer 
Evans et al. 
Foreman & Kirchhoff 
George & Robbins 
Goldberg et al. 
Goodrich & Roland 
Hinchcliff et al. 
Kristof 

McLellan et al. 
Nuckles et al. 
Pandit 
Pope 
Poyer 

Putterman & Lossos 
Stull et al. 



A Comparison of Citation Accuracy Studies 



Subject Scope 

Library & Medical Sciences 

Social Sciences 

Medicine 

Dentistry 

Public Health 

Surgery 

Nursing 

Dermatology 

Emergency Medicine 

Medicine 

Veterinary Medicine 
Entomology 
Anesthesiology 
Dentistry 
Library Science 
Library Science 
General Sciences 
Medicine 

Physical Education 



Dalfi #C»atlons examined 



1993 
1979 
1985 
1988 
1987 
1990 
1987 
1994 
1993 
1977 
1993 
1995 
1992 
1993 
1993 
1992 
1979 
1991 
1991 



525 

1,012 
300 
500 
150 
150 
1 1 2 
240 
145 

2,195 
295 

1,337 
348 
298 

1,094 
100 

2,448 
384 
973 



Most errors In 

authors' names 

numerical data 
• 

article titles 
authors' names 



authors' names 

article titles 
authors' names 
article titles 
article titles 
article titles 
numerical data 
authors' names 
authors' names 
authors' names 
authors' names 



AVERAGE ERROR RATE FOR ALL STUDIES 

* Data not available from study. 



% Errors 

29% 

13% 

24% 

42% 

31% 

48% 

30% 

41% 

28% 

29% 

30% 

30% 

50% 

21% 

18% 

30% 

15% 

33% 

47% 

31% 
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Table 5: 



Comparison of Citations to Journal Articles with Citations to Books 



Archives Insect Biochemistry and Physiology 
Journal articles 
Books 



Verified Containing Errors 



130 
24 



38 
5 



% Errors 

29% 
21% 



Ecological Entomology 

Journal articles 
Books 



310 
61 



76 
22 



25% 
36% 



Insect Biochemistry and Molecular Biology 
Journal articles 
Books 



281 
28 



91 
1 7 



32% 
61% 



Journal of I nsect Physiology 

Journal articles 
Books 



209 
39 



61 
19 



29% 
49% 



Pesticide Biochemistry and Physiology 
Journal articles 
Books 



201 
40 



49 
20 



24% 
50% 



Average percentage of citations containing errors 
Average percentage of citations containing errors 



journal articles 
books 



28% 
43% 
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bibliographic information or "elements" than journal citations. For example, if an 
author cites a chapter from a multi-volume set of books, nine minimum elements 
are needed in the citation: chapter author, chapter title, book editors, book title, 
place of publication, publisher, volume number, year, and page numbers. In 
contrast, when an author cites the entire book, only six elements are required, 
which is the same amount required for a journal article citation. The data seem to 
support this. Out of the 192 book citations in this study, 137 of them were citations 
to chapters within the books. Sixty-six (48%) of these contained one or more errors. 
However, of the 55 citations to entire books, proportionately fewer (17 or 31%) 
contained one or more errors. Another possible reason for this discrepancy is 
because books are simply cited less often than journals; authors get more practice 
with citing journal articles. 

Out of the 49 articles in the five journal issues examined, only three articles 
had completely error-free reference lists. One error-free reference list each appeared 
in Journal of Insect Physiology, Insect Biochemistry and Molecular Biolog y, and 
Ecological Entomology . While, the researcher was unable to contact the editors of 
all the journals studied, the researcher spoke with an editor of lournal of Insect 
Physiology, who said that while reference lists were examined for content, each 
reference was not examined for accuracy. Judging from the similarity of errors and 
error patterns among these journals, it is likely that none of the editors of these five 
journals examines references for accuracy during the editorial process. Authors of 
previous studies have suggested that authors should be primarily responsible for 
the accuracy of their bibliographies, however, it has been noted also that editors, 
reviewers, or publishers should attempt to verify at least a small portion of the 
references in papers accepted for publication. Doms' suggestion that editors or 
editorial staff "make a policy of checking 10% the citations for each article as part of 
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the acceptance process" (1989, p. 444) still seems to be an idea worth implementing. 
Significance: 

Previous bibliographical citation accuracy studies have examined primarily 
medical and library science literature. Studies of the literature of other fields is 
lacking. This study was based, in part, upon previous studies of this nature. It is 
important to determine if bibliographical references in the literature of other fields 
is as error-ridden as the literature that has been studied thus far. This study also 
sought to determine what kind of publications authors make the most errors in 
citing. 

This study examined the field of entomology only; there are many other 
fields that have yet to be examined, for example education and other social sciences. 
Further studies of this sort might survey editorial boards of journals to find out if 
any amount of references are verified before publication. Also, further studies 
might attempt to find out whether or not there is a correlation between citation 
information printed on the first page or all pages of the cited article and citation 
accuracy. Further studies might also examine the different types of article title or 
author name errors in greater detail to try to determine if any patterns appear that 
could shed some light on why these errors continue to be the most frequent. 

Bibliographical references are used for many purposes, such as document 
delivery, document location, bibliometric and citation studies, and evaluation of an 
author's work. Accurate references facilitate all of these activities. Accurate 
references demonstrate the kind of attention to detail that should be a part of every 
scholar's research. 
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